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Abstract 

This paper investigates the optimal harvesting strategy for a single species living 
in random environments whose growth is given by a regime-switching diffusion. Har- 
vesting acts as a (stochastic) control on the size of the population. The objective is to 
find a harvesting strategy which maximizes the expected total discounted income from 
harvesting up to the time of extinction of the species; the income rate is allowed to be 
state- and environment-dependent. This is a singular stochastic control problem with 
both the extinction time and the optimal harvesting policy depending on the initial 
condition. One aspect of receiving payments up to the random time of extinction is 
that small changes in the initial population size may significantly alter the extinction 
time when using the same harvesting policy. Consequently, one no longer obtains con- 
tinuity of the value function using standard arguments for either regular or singular 
control problems having a fixed time horizon. This paper introduces a new sufficient 
condition under which the continuity of the value function for the regime-switching 
model is established. Further, it is shown that the value function is a viscosity solu- 
tion of a coupled system of quasi-variational inequalities. The paper also establishes 
a verification theorem and, based on this theorem, an e-optimal harvesting strategy 
is constructed under certain conditions on the model. Two examples are analyzed in 
detail. 
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1 Introduction 



One of the most important yet difficult problems in modern natural resources management is 
the establishment of ecologically, environmentally and economically reasonable wildlife man- 
agement and harvesting policies. There are many occurrences where myopic unconstrained 
harvesting has led to local and/or global extinctions. Lande et al. [TT] documents many such 
examples. Real ecological communities are random by nature. As a result of developments 
in stochastic analysis and stochastic control techniques, there has been a resurgent interest 
in determining the optimal harvesting strategies in the presence of stochastic fluctuations 
(see, e.g., [21 HJ HI EH])- Unfortunately, most of the current research on harvesting problems, 
including the aforementioned references, are primarily focused on a single species in a static 
environment. The paper by Lungu and 0ksendal [IB] makes a first step in the analysis 
of the harvesting problem for interacting populations but does not consider changes in the 
environment. 

As noted in [61 [131 EI], the variations in the external environment (for example, weather 
or anthropogenic) can have important effects on the dynamics of the populations of the 
ecosystem. In addition to the random fluctuations of the populations (usually modeled by 
white noise; see, e.g., [3]), certain biological parameters such as the growth rates and the 
carrying capacities often demonstrate abrupt changes due to environmental noise. Moreover, 
the qualitative changes of those parameters form an essential part of the dynamics of the 
ecosystem. For example, Medina-Reyna [22J demonstrates that the mean growth rates of 
white shrimp (Litopenaeus vannamei) in the Mar Muerto Lagoon, Southern Mexico are 
significantly different in various salinity levels. Similar observations were made in [26] for 
reproduction performance of crossbred goats in a derived Guinea savanna zone. For another 
example, the carrying capacities often vary according to the changes in nutrition, water 
supply, living spaces, and/or food resources (see [30] for many such examples). In the 
mathematical community, people are paying more attention to the modeling and analysis 
of population dynamics subject to both white and colored noises; see, e.g., [3 [201 EH] and 
references therein. 

Naturally, one expects that the optimal harvesting strategies may vary according to the 
changes of the environment. Despite the increasing interests in the mathematical modeling 
and analysis of population dynamics, to our best knowledge, there are relatively few results 
in the literature that address harvesting strategies in random environments. This paper 
addresses this hole in the literature by examining optimal harvesting problems of a species 
in random environments. 

Suppose there is a single species whose growth is subject to the usual fluctuations as well 
as the abrupt changes of the random environments. Harvesting strategies are introduced 
to derive financial benefit as well as to control the growth of the population. The goal is 
to find a harvesting strategy which maximizes the expected total discounted income from 
harvesting, up to the time when the population falls to a given threshold (e.g., extinction). 
Harvesting may occur instantaneously so results in a singular stochastic control problem in 
the sense that the optimal harvesting strategy may not be absolutely continuous with respect 
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to the Lebesgue measure of time. In other words, in contrast to the regular stochastic control 
problems, in which the displacement of the state due to control is differentiable in time, the 
harvesting problem considered in this work allows the displacement to be discontinuous. This 
paper establishes a verification theorem and, based on the theorem, explicitly constructs an 
£-optimal harvesting strategy. Both the extinction time and harvesting policy may depend 
on the initial conditions. As a result, continuity of the value function can not be obtained 
using the standard arguments for regular or singular stochastic control problems in a fixed 
time horizon. This paper provides a sufficient condition under which the continuity of the 
value function is guaranteed. It is further shown that the value function is a viscosity solution 
of a coupled system of quasi- variational inequalities (12. ip . 

The novelty of this work arises in two distinct ways. The modeling of random environ- 
ments through the use of a continuous-time finite-state Markov chain introduces coupling of 
the value function for each environment in the quasi-variational inequalities for the verifica- 
tion theorem (Theorem I2.ip . An ^-optimal harvesting policy (Theorem 12. 4p is determined 
under certain conditions that involves quickly harvesting very small amounts until the species 
becomes extinct in a very small time interval. We should also remark that the proof of The- 
orem 12.41 is very technical and non-trivial. In addition to the subtle analysis in dealing with 
the controlled process X, the presence of environmental switching adds much difficulties in 
the proof. The introduction of different regimes necessarily implies that the optimal harvest- 
ing strategy will depend on the current environment (Example 13. ip . The determination of an 
environment-dependent optimal policy is non-trivial as one must overcome some significant 
technical challenges. 

This paper's second contribution comes from identifying a new sufficiency condition for 
the continuity of the value function as a function of the initial state. The fact that payment 
is received only until the random time of extinction and this time strongly depends on the 
harvesting policy adopted means that a small decrease in initial population size may result 
in a significant decrease of the extinction time. Continuity is therefore not a direct extension 
of standard results for a fixed time horizon. Theorem 14.41 establishes a sufficient condition 
under which the value function can be proven to be continuous for this criterion involving the 
hitting time of the population at (or any quasi-extinction level from which the population 
will not rebound). Once the value function is shown to be continuous, it is then proven to 
be a viscosity solution of the quasi-variational inequalities (Theorem 14. 9p ; even this analysis 
is technically challenging due to the existence of multiple environments. 

Note that this work is expressed entirely in terms of harvesting of a single species in ran- 
dom environments, but as in Miller and Voltaire [23] , the harvesting problem is a paradigm 
that has many additional economic applications. 

Besides the optimal harvesting problems considered in this paper and [2j [THJ HH] , singular 
stochastic control has found applications in many other areas. For example, singular stochas- 
tic control problems naturally arise in monotone follower problems [H], optimal dividend 
distribution schemes [3 [27], portfolio selection management with transaction cost [2f~j 125]. 
diffusion control of many-server queues [33], and heavy traffic modeling and control prob- 
lems [31]. We refer the reader to [HI [36] for more such examples. See also [TU [12] for 
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a general singular stochastic control problem for a multidimensional Ito diffusion on a fixed 
time horizon, in which the existence of the optimal control and the characterization of the 
value function as the unique viscosity solution of a Hamilton- Jacobi-Bellman equation are 
established. Most, if not all, of the existing literature on singular stochastic controls consider 
with Ito (jump) diffusions. 

Regular control and optimal stopping problems for regime switching diffusions have be- 
come more popular recently (see, for example, [HI [TOl E21 137] and references therein). Less is 
known for singular control of regime switching diffusions. Moreover, a common assumption is 
that the marginal yield from exerting the singular control is constant. Two exceptions are in 
PQdH], where the marginal yields depend on state and time, respectively. The assumption of 
constant marginal yield seems rather restrictive since in the real world, the unit price usually 
depends on the current state of the system. This paper considers state-and-regime-dependent 
marginal yields from harvesting; that is, the unit price depends on both the current state of 
the population size of the species and the regime of the environment. This additional feature 
of the model is not merely an extension of the traditional models, but in fact, introduces 
many interesting mathematical problems for the analysis. More specifically, there may not 
exist admissible optimal harvesting strategy under this setting; see Theorem I2.4[ Remark 
13. 2^ and Example 13.31 for more details. Nevertheless, using detailed and careful analysis of 
the sample path properties of the controlled process, we constructs an explicit admissible 
e-optimal harvesting strategy. 

The rest of the paper is organized as follows. A precise formulation of the problem is 
presented in Section 11.11 Then a verification theorem is proven in Section [2] and is used 
to explicitly construct an e-optimal harvesting strategy under additional conditions. Two 
examples are given in Section [3] to illustrate these results. Section H] derives the continuity 
of the value function V and characterizes it as a viscosity solution of a coupled system of 
quasi- variational inequalities (12. ip . Section [5] contains concluding remarks. 

A few words about notation is needed. A function from [0, oo) to some Polish space E 
is cadlag if it is right continuous and has left limits in E. When E = R and £ is cadlag, 
then A£(i) =£(£)— £{t— ) for t > and the convention A£(0) = £(0) is used. As usual, 
sup = — oo and inf = +oo. For any a, b e R, a + = max {a, 0} and a A b = min {a, b}. If 
B is a set, Is denotes the indicator function of B. 

1.1 Formulation 

Suppose a certain species, whose population size at time t is denoted by X(t), lives in 
random environments. As alluded in Section [H in addition to the random fluctuations of the 
population, we also assume that the growth of the species is subject to abrupt changes of 
the environment. For simplicity, we assume that the switching among different environments 
is memoryless and the waiting time for the next switch is exponentially distributed. In 
fact, this phenomenon is also frequently observed in the nature; see the aforementioned 
references. Thus we can model the random environments and other random factors in the 
ecological system by a continuous-time Markov chain {a(t),t > 0} with a finite state space 
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M. = {1, . . . , m}. Let the continuous time Markov chain a(-) be generated by Q = 
that is, 

„ r , .x /x / \ I QaAt + o(At), if j ^ i , 

P{a(t + At)=j\a(t)=z,a(s),s<t} = {^ 1 > JT . (1.1) 

[1 + quAt + o(At), if J =i, 

where qij > for i, j = 1, ... ,m with j ^ i and qa = — Qij < f° r eacn * = 1) • • • > m - 

In light of the above discussion, in an effort to capture the salient feature that continuous 
dynamics and discrete events coexist in the ecosystem, we model the evolution of X(t) in 
the absence of harvesting by the stochastic differential equation 

dX(t) = b(X(t),a(t))dt + cr(X(t),a(t))dw(t), X(0) = x, a(0) = a, (1.2) 

where w(-) is a 1-dimensional standard Brownian motion which provides the random fluc- 
tuations in the population's size, and b and a are real-valued functions. Further, we assume 
that the Brownian motion w(-) and the Markov chain a(-) are independent, a standard 
assumption in the literature. 

Assume throughout the paper that b and a satisfy the usual Lipschitz condition and the 
linear growth condition. That is, there exists some k > such that for any x, y G K and 
each a 6 M, we have 



\b(x, a) — b(y, a)\ + \cr(x, a) — a(y, a)\ < k \x — y\ 
\b(x,a)\ + \a(x,ot)\ < k (1 + \x\). 



;i.3) 



Consequently, the solution X x ' a (-) of CI 1 . 2 1) exists and is unique in the strong sense (see [35] 
for details). Moreover, the solution X x,a (-) will not explode in finite time with probability 1 
or it is regular in the sense of Khasminskii [15J . We refer the reader to [32] for related results 
on the regularity of regime switching diffusions. 

If the species is subject to harvesting, and if Z(t) denotes the total amount harvested 
from the species up to time t, then X(-), the population size of the harvested population, 
satisfies 

dX(t) = b(X(t), ot{t))dt + a{X{t), a{t))dw{t) - dZ{t), (1.4) 
with initial conditions 

X(0-)=xGR+, a(0)=aeM. (1.5) 

Note that X(0) may not equal to X(0—) due to an instantaneous harvest Z(0) at time 0. 
Throughout the paper we use the convention that Z(0—) = 0. The jump size of Z at time 
t > is denoted by AZ(t) := Z(t) - Z(t-), and Z c (t) := Z{t) - J2o<s<t AZ ( s ) denotes the 
continuous part of Z. Also note that AX(t) := X(t) — X(t—) = —AZ(t) for any t > 0. 
Denote the solution to (11.41) with initial condition specified by (11.51) by X x,a {-) if necessary. 

We say that Z is an admissible harvesting strategy if 

(i) Z{t) is nonnegative for any t > and nondecreasing with respect to t, 

(ii) X(t) > 0, for any t < r, where r is the extinction time defined in (11.61) below, 
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(iii) Z(t) is cadlag and adapted to J c t '■= c { w ( s ), a ( s ), < s < t}, and 

(iv) J(x, a, Z) < oo for any x > and a £ A4, where J is the functional defined in (1 1.7ft 
below. 

Let A denote the collection of all admissible harvesting strategies. 

Let /(•,•) : K+ x M 4 M + represent the instantaneous marginal yields accrued from 
exerting the harvesting strategy Z. Assume / is continuous and non-increasing with respect 
to x. Thus f(x, a) > f(y, a) for each a G M. whenever x < y. Moreover, we assume 
< /(0, a) < oo for each aGM. Let 5 = (0, oo), which may be regarded as the survival 
set of the species. Denote the extinction time by 

r := r x ' a = inf |t > 0, X x ' a {t) £ s} . (1.6) 

Then for a fixed harvesting process Z G A, the expected total discounted value from har- 
vesting is 

J(x,a,Z) := B Xia f e~ rs f{X( S -),a(s-))dZ(s) = E f e~ rs f(X*> a {s-), a(s-))dZ(s), 
Jo Jo 

(1.7) 

where r > is the discounting factor and denotes the expectation with respect to the 
probability law when the process (11.41) starts with initial condition (x, a) as specified in (II. 5p . 
The goal is to maximize the expected total discounted value from harvesting and find an 
optimal harvesting strategy Z*: 

V(x, a) = J(x, a, Z*) := sup J(x, a, Z). (1.8) 

ZeA 

The dynamic programming principle takes the form (see [81 128]): 



V(x, a) = sup E Xia 
ZeA 



Uo 



e- rs f(X x ' a (s-),a(s-))dZ(s) 
+ e - r(TAr,) V(X x ' a (T A 77), a(r A 77)) 



1.9) 



for every (x,a) E S x Ai and any stopping time 7]. 

For later convenience, we introduce the generator of the paired process (X x,a , a), in which 
X x ' a satisfies (jl~2l) . For any h(-,a) G C 2 , a G M, we define 

Ch(x, a) = b(x, a)h'(x, a) + -a 2 (x, a)h"(x, a) + q a j[h(x, j) — h(x, a)}, 

where h! and h" denote the first and second order derivatives of h with respect to x, respec- 
tively. 
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2 Verification Theorem and an e-Optimal Policy 



This section establishes a verification theorem whose proof utilizes the generalized Ito for- 
mula, the monotonicity of /, and the regularity of the process X x,a . We further construct an 
e-optimal harvesting strategy explicitly in Corollary 12.41 based on the verification theorem 
and the imposition of additional conditions. 

Theorem 2.1. Suppose there exists a function <p : SxJ\4 i— > M. + such that<f)(-, a) G C 2 (S) for 
each a G Ai and that solves the following coupled system of quasi-variational inequalities: 

max {(£ — r)<p(x, a), f(x, a) — (p'(x, a)} = 0, (i,«)g5xM, (2.1) 

where (£ — r)0(x, a) = C(f)(x, a) — r<p(x, a). 

(a) Then a) > V(x, a) for every (x, a) G S x M,. 

(b) Define the continuation region 

C = {(x, a) G S x M : f(x, a) - <fi'(x, a) < 0} . 

Assume there exists a harvesting strategy Z G A and corresponding process X satisfying 
(11.41) such that, 



(X(t), a(t)) G C for Lebesgue almost all < t < 



4>'{X(s), a(s)) - f(X(s), a(s)) dZ c (s) = 0, for any t < r, 



lim E Xta 

AT->oo 



-r(^NAM^x(T ANA /3jv), a(r ANA (3 N )) 



0. 



and if X(s) ^ X(s-), then 

<f>(X(s), a(s-)) - <f>(X(s-), a(s-)) = -f(X(s-), a(s-))AZ(s), 



(2.2) 
(2.3) 
(2.4) 

(2.5) 



where (3^ '■= inf{t > : |X(t)| > N}. Then (j)(x, a) = V(x, a) for every (x,a) G SxAi 
and Z is an optimal harvesting strategy. 

Proof, (a) Fix some (x, a) G S x M. and Z G A and let X denote the correspond- 
ing solution to (II. 4p . Choose N sufficiently large so that \x\ < N and define f3jq : = 

inf ji > : \X(t)\ > Jvj. By virtue of [351 Section 2.3], 

(2.6) 



Write T, 



N 



f3j\f — y oo a.s. as N — > oo. 
N A f3]y A t. Then Ito's formula leads to 

-rT Nr 



E rn [e~ rTN (l)(X{T N ), a(T N ))] - <f>(x, a) 



E, 



+E :Cia e 

0<s<T N 



e- rs (C-r) ( j ) (X(s),a(s))ds-E X: 

4>(X(s),a(s-)) - cf>(X( S -),a(s-)) 



e- rs <j)'{X{s),a{s))dZ c {s) 
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It follows from (EE} that 

E Xja [e- rTN (f)(X(T N ), a(T N ))} - a) 

fT N 

<-E x , a e- rs <l ) '(X(s),a(s))dZ c (s) + E x ^ ^ e~ rs A<f)(X (s) , a(s-)), 



o 



0<s<T, 



where A(f)(X(s), a(s— )) = <f>(X(s), a(s— )) — 4>(X(s— ), a(s— )). Apply the mean value theo- 
rem to A(j)(X(s), a(s— )) and we obtain 

A<p(X(s),a(s-)) = </>XZ{s),a{s-))AX(s) = a(s-))A£(s), 

where = 0(s)X(s) + (1 - 6(s))X(s-) for some 0(s) G (0, 1). Note that X(s) < £(s) < 
X(s-). Thus it follows that 

<j>(x,a) >E x ^ a [e- rTN <P{X{T N ),a{T N ))] + E x , a / e^^ 1 {X{s), a{s))dZ\s) 

Jo 

+ E Xia e- rs ^(Z(s),a(s-))AZ(s). 

0<s<T N 

Using (12. ip again and noting that <fr is nonnegative and that /(-, a) is nonincreasing for each 
a e M.. it follows that 



T. 



{x, a) > E x , a / e- rs f(X(s), a(s))dZ%s) + E x>a ^ e^f^s), a(s-))AZ(s) 

0<s<T N 

fTN 

>E x , a e- rs f(X(s),a(s))dZ c (s) + E x , a ^ e~ rs f(X(s-), a{s-))AZ{s) 

^0 n^o^T".. 



0<s<T N 
fTN 

= E X , Q / e - rs f(X(s-), a(s-))dZ(s). 
Jo 

Now letting N — > oo, it follows from ( 12. 6 p and the bounded convergence theorem that 

<f>{x,a) > E X , Q [ T e- rs f(X(s-),a(s-))dZ(s). 
Jo 

Finally, taking supremum over all Z e A, we obtain a) > V(rr, a), as desired. 

(b) Let Z G A satisfy (12.21) ( T2~5|) . Define (3n and Tjy as before with X replacing X. As 
in part (a), we have from Ito's formula that 

E,, a [e- rT -0(X(T w ), a(T N ))} - a) 

fTN fT N ^ 

= E x , a / e~ rs (C - r)0(X(s), a(s))ds - E X , Q / e"^' (X( S ), a(s))dZ c (s) 
Jo Jo 

+ E x , a e- rs U(X(s),a(s-))-<l ) (X(s-),a(s-)) 



0<s<T 



N 
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By Q, (£ - r)(j)(X(s),a{s)) = for almost all s G [0,r]. This, together with Q and 
(12. 5p . implies that 

_ /"Tat _ _ 

<j>(x,at) = ^ a [e- rTN <P{X{T N ),a{T N ))} + / e - rs / (X(s-), a(s-))dZ(s). 
Letting AT — > oo and using (12. 4p and ( 12 .6 j) . we obtain 

0(x,a) = E Xia [ T e- rs f(X(s-),a(s-))dZ(s). 



This shows that 0(a;, a) = V(x, a) for every (x,a) <E S x M and Z is an optimal harvesting 
strategy. □ 



Remark 2.2. The conditions of Theorem 12.11 can be weakened. In fact, by virtue of [21 
Appendix D], we need only to assume that (i) </>(•, a) G C 1 (5) (H C 2 (5 — D) for each a G .M, 
where .D is countable set of points, and (ii) (f)"(x+) < oo, (f>"(x—) < oo for all x G D. Under 
these conditions, there exist sequences {</>j(-, a e M such that (fij(-,a) G C 2 (S) for 

each a G A1. Moreover, the following are satisfied: 

(a) for each a G M., lim^oo 4>j{-, a) — > a) uniformly on compact subsets of S, 

(b) lim :; _ >00 (£ — r)(f>j(x, a) —> (f>(x, a) uniformly on compact subsets of S — D, a G M. , and 

(c) {(£ — r)(j)j}™ =l is locally bounded on S x .M. 

Then, we can first work with the sequence <pj exactly the same way as in the proof of 
Theorem 12. 1[ Next, using (a), (b), and (c), we can pass to the limit as j — > oo to obtain the 
same conclusions. The reader is referred to [Ml for details. 



By virtue of Theorem 12 .1( a). any sufficiently smooth solution to (12. ip is an upper bound 
for the value function V. Further, the additional conditions in Theorem 12.1( b) will help us 
to find an optimal harvesting strategy. In practice, it is, however, usually very hard to find 
an explicit solution to (12. ip . In particular, with the presence of regime switching, ([2.1 p is a 
coupled system of quasi- variational inequalities, a closed form solution is virtually impossible 
except in some special cases (see Examples 13 .11 and 13. 31) . Nevertheless, some results about the 
value function can be derived; Proposition !2.3l below gives an upper bound for V when /(-, a) 
is smooth for each a. Furthermore, under additional assumptions, we explicitly construct 
an e-optimal harvesting strategy in Theorem 12.41 

For any x > and a G M, define 

g{x,a) = / f(y,a)dy. (2.7) 
Jo 

Then it follows that g is nonnegative and g'(x,a) = f(x,a). Moreover, if /(-,«) G C 1 (S'), 
then g"(x,a) = f'(x,a) < because f(-,a) is nonincreasing for each a G M.. This shows 
that g(-,ct) is concave for each a G M.. 
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Proposition 2.3. Assume that f{-,oe) G C 1 (S') and f(-,a) is non-increasing for each a G 
Jvt. Then we have 



V(x, a) < g(x, a) + sup E 2 



e- rs (C-r)g(X(s),a(s))ds. 



(2.1 



Proof. Fix some (x, a) G S X Ai and Z e A and let X denote the corresponding solution to 
(11.4p . Let Tjv be as in the proof of Theorem 12.11 Apply Ito's formula using g to obtain 



Ex., 



Tat 



E 



e- rs (£ - r)</(X(s), a(s))ds - E X , Q / e^V (X(s-), a(s-))dZ(s) 



o 



+ Ea., a > 

0<s<T N 



E 



9 (X(s), a(»-)) - s(X(s-), a(s-)) - s'tXfs-), a( s -))AA'( s )] 



Since g(-, a) is concave for each a G .M, it follows that 

</(*(*), a(s-)) < g(X(s-), a(s-)) + </(X(s-), a(s-))(X(s) - X(s-)). 
Thus we have 

g(x, a) > E X , Q / e~ rs f(X(s-), a(s-))dZ(s) - E x>a / e - rs (C - r)g(X(s), a(s))ds. 



Now letting N — > oo and using the same argument as in the proof of Theorem 12. 1[ we obtain 



g(x,a) > E. 



,,„ , e' rs f(X(s-),a(s-))dZ(s)-E x , a / e- rs (£-r^(X( S ),a( S ))rf S , 
o Jo 



from which ( 12. 8ft follows by taking supremum over Z G A. 



□ 



Theorem 2.4. Assume, in addition to the conditions of Proposition |23| (£ — r)g(x, a) < 
/or a// (x, a) & S X M. 

(i) Suppose that there exists a constant L > s«c/j t/iat 

a) — f(y, a) \ < L \x — y \ , for all x,y G S and a G M.. (2.9) 

TVien for any e > 0, t/iere exists a harvesting strategy Z £ E A under which 

g(x, a) - e < J(x, a, Z e ) < a) < #(x, a). (2.10) 

The harvesting strategy Z e is a "chattering policy" that instantaneously harvests a 
sufficiently small amount many times in a sufficiently small interval of time until the 
species becomes extinct. 

(ii) In particular, if f(x, a) = f(ct) for all x G S and each a G M., then 

V(x,a) = g(x,a) = f(a)x, (2-11) 
and the optimal harvesting strategy is to drive the process instantaneously to extinction. 
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Proof. Fix some (x, a) G S x .M. By Proposition 12.31 and the condition (£ — r)g(x, a) < 0, 
we have 

V(a;,a) < g(x,a). (2.12) 

The rest of proof is divided into two parts. Part 1 is devoted to the proof of (12.101) while the 
second part establishes (12.111) . 

Part 1. Since /(-, a) is continuous, for any e > 0, there exists an iV e N such that 

n-l . x 

R(f,a) :=y2f(xi,a)5 > / f(y, a)dy — e/3 = g(x, a) — e/3, for any n> N, 

where 5 = x/n and x% = x — i8, i = 0, 1, . . . , n — 1. Note that we also have R(f, a) < g(x, a) 
since /(•,«) is non- increasing. Thus it follows that 

\R(f,a)-g(x,a)\ < e/3. (2.13) 

Let q = n~ 5 and ^ = iq/n, i — 0, 1, . . . ,n. We construct a harvesting strategy Z = Z e 
which increases only on the set {tj : z = 0, ...,n}; denote the corresponding harvested 
process by X. Note that X(to— ) — Xq — X. Define AZ(t ) = Z(t ) = 5, observe X(t ) = x x 
and it therefore follows that 

X(t 1 -) = X(t )+ f 1 b(X(s),a(s))ds+ I * a(X(s),a(s))dw(s). 

J to J to 

At time t = t±, define AZ(ti) = (X(ti—) — x 2 ) + so that X(tx) < x 2 and allow the process 
X to diffuse until time t = t 2 . In general, for i = 1, . . . , n — 1, we define 

AZ(U) = (X(ti-) - x i+1 
so that 

X{t i )=X{t i -)-AZ(t i ), 

and t t 

X(t i+ i-) = X(t i )+ [ ' +1 b(X(s),a(s))ds+ I ' ' +± a(X(s), a(s))dw(s). 
Jti hi 

Note that X(ti) = x i+ \ if AZ{pi) > 0. The expected total discounted income from the 
harvesting strategy Z is 

n-l 

J(x, a, Z) = E x>a e- rti f{X{U-), a(t i -))AZ(t < ). 

i=0 



11 



Next we want to show that \ J(x, a, Z) — R(f, a)\ < e/3. In fact, we have 



n-l 



\J(x,a,Z)-R(f,a)\ <^E XiQ f (X fa-) , afa-)) AZfa) - f(x u a) 6 



i=0 
n-l 

i=0 



E, 



[f(Xfa-), afa-)) -f{x h a)\8 

+ \f(Xfa-),afa-))[AZfa) - 5] 
+ ~E Xta | [e~ rti - l]f(Xfa-),afa-))AZfa 

n 



(2.14) 



8=1 



In the following, we analyze the terms Aj, B iy and Cj separately. To this end, for any 
i = 0, 1, . . . , n — 1, we apply [351 Proposition 2.3] to obtain 



E 



b(X(s),a(s))ds + / cr(X(s), a(s))dtt;(s) 



E|AZ(ti)|=E (Xfa-)-x i+1 



< K, 



<Kfa +1 -t i ) = Kt 1 , (2.15) 

(2.16) 



where K is a generic positive constant depending only on x, m, and the constant k,q in 
(11.31) . Also, in the sequel, the exact value of K may change in different appearances. Then 
it follows from the Tchebychev inequality that 



P {AZfa) = 0} = P <^ Xfa-) < x 2 



< P 



to 



b{X{s),a{s))ds + / <r(i"(s), a(s))dw(s) < -5 



to 



to 



b(X(s),a(s))ds+ / a(X(s),a(s))dw(s) 



> S 



(2.17) 



< 



d 2 



Note that Xfa) = x 2 if AZ(ii) > 0. Thus we have 

v{xfa)^x 2 \<V{AZfa) = $\< 



Using the same arguments as those in ( 12.171) and ( 12.181) . we have 

P {AZfa) = 0} = P ^AZ(t 2 ) = 0,X(h) = x 2 } + P {AZfa) = 0, Xfa) ^ x 2 

Kti Kt x Kt 2 
< — - + 



(2.18) 



(2.19) 



5 2 5 2 5 2 ' 



and 



{xfa)^x 3 }<P{AZfa) = 0}< 



\2 • 



(2.20) 
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Continuing in this manner, it follows that for any i = 1,2, . . . ,n — 1, 



P{AZ(t i )=0}< 



S 2 ' 



P X(U) ± < 



and 
S 2 ' 



(2.21) 
(2.22) 



Using the conditions that / is Lipschitz continuous and uniformly bounded, we compute 



A<E f(X(t i -),a)-f{x i ,a) 

< LE X(U-) - x 

< LE X(U-) - x 



6 + E 

8 + KP ) ^ a} 5 

8 + Ktid, 



f(X(ti-),a{ti-))-f(X(ti-),<x) 



where in the last inequality, we used (jl.ip . But using (|2.22p . (12.151) . and (35], Proposition 
2.3], we obtain 



E 



X{ti-) - Xi 



< E 



< 



Thus it follows that 



< E 1 ' 2 



X{ti-i) - x 



+ E 

2 



ti-1 



b(X(s),a(s))ds + / a(X(s),a(s))dw(s) 



5 



+ Ktl < E^E + Ktl . 

o 



(2.23) 



Ai < K5 (^j- + U + tij = K(Vti + US + kS). 

Next we estimate Bi. Since / is uniformly bounded, it follows that 
Bi < KE \AZ(U) - S\ 

= KE\(AZ(U) - 5)I {AZ(U)=0} \ + KE\(AZ(U) - S)I {AZ(ti)m I {x(ti _ l)=x A 

+KE | (AZ(ti) - S)I {AZ{u)m I {Mu i) ^ } \ 
:= Bn + B i2 + B i3 . 

Note that fl2T2"B implies that B a < 5^- = Using the definition of AZ(U) and (PXToD . 
we have 



B l2 = KE 



ti-i 



(X(ti-) - X i+1 - 5)i'{AZ(t i )#0}^jt( t4 _ l)=Xi } 

a(X(s),a(s))dw(s) J I{Az( ti )^o}I{x {t ._ l)=x .y 



= KE 

Concerning the term B i3 , we use the Cauchy-Schwartz inequality, (12.161) . and f!2.22[) : 



B t3 < KE X I 2 \ (AZ(ti) - 5)I {Azik)m \ 2 E 1 / 2 



■{x(U-i)^ Xi } 



5 
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Putting these estimates together, we obtain 

B l <K% + Kti + < K(t x + ^5). (2.24) 



For the term Cj, we again use the uniform boundedness of / and (12.161) to obtain 



{e~ rU - l)f(X(ti-), a(ti-))AZ(ti) < K(l - e~ rU ) = Krt t + o{U) < Kt,. (2.25) 



Now using the estimates (T2T23]) . f[2T24"j) . and ([2J23D in (T2~T4l . and noting 5 = xn~\ U = it u 
and t\ = n~ 6 , 



\J(x, a 



Z) - R{f, a)\<J2 (k(VU + U5 + h5) + K(h + ^) + Kt^j 

i=i ^ 

<K[t 1 ^{i + l) + ^/t l 5 -^Y,Vi 



6 

i=l i=l 



< fsA- 6 + n~ 3 nn 3/2 < Kn- l/2 . 



Finally we choose n sufficiently large so that (12.1 3p holds and \J(x, a, Z) — R(f, a)\ < e/3. 
Then it follows that 

\J(x, a, Z) — g(x, of) | < \J(x, a, Z) — R(f, at)\ + \R(f, a) — g(x, a)\ < e/3 + e/3 < e. 
Now (T2~T0]) follows in view of (12TT21 . 

Part 2. If f(x,a) = f(a) for all x G S and each a G M., then we choose Z to be the 
harvesting policy which drives the process X instantaneously from state x to state 0. It 
follows that r = and 

J(x, a, Z) = f(a)x = g(x, a) = V(x, a). 
This finishes the proof. □ 

Remark 2.5. In [U Corollary 1], it was commented that "if the convenience yield from 
holding reserves is non-positive at all states then the optimal policy is to deplete the reserves 
at an infinitely fast rate but only in small proportions at a time (a form of a 'chattering 
policy')." While the intuition in [TJ is correct, the optimal policy in [1] is not admissible in our 
context, because it is not well defined at time 0. In Theorem 12 A\ we explicitly constructed 
an admissible harvesting policy, under which the expected total discounted income from 
harvesting is ^-optimal. 



3 Examples 

We provide two examples to demonstrate our results in the previous section. They reveal 
that in the setting of regime switching, it is much harder to obtain the value functions and 
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the (e-) optimal harvesting strategies due to the coupling in the system of quasi variational 
inequalities. 

Example 3.1. We assume the growth of a certain species (or a certain risky investment) is 
governed by a regime switching geometric Brownian motion 

dX(t) = /i{a{t))X(t)dt + a(a{t))X{t)dw{t), (3.1) 

and the harvested process is given by 

dX(t) = fx(a(t))X(t)dt + a(a(t))X(t)dw(t) - dZ(t), (3.2) 

where Z(t) denotes the total amount of harvest (or dividends) up to time t, w is a standard 
Brownian motion, a is a continuous time Markov chain with state space M. — {1, . . . ,m}, 
and for each a 6 M, |i a = /z(a) and a a = a(a) are constants. Our objective is to maximize 
the expected discounted income from harvest and find an optimal harvesting policy, i.e., we 
want to find 



V(x 1 a) = J(x, a, Z*) = sup E XjQ , / e rs dZ(s), (3.3) 

zeA ' Jo 

where r > is the discount factor. Note that / = 1 in this example. 

First consider the case when m — 1; that is, there is only a static environment so no regime 
switching occurs. It is clear that if /j, > r, then V(x, 1) = oo. On the other hand, if \i < r, 
then V(x, 1) = x and the optimal harvesting policy is to drive the process instantaneously 
to extinction (r = a.s.). We refer the reader to [1] or [2] for details. 

Now let m — 2 and assume that the continuous time Markov chain a is generated by 
Q = ( Z^ 1 } ), where Ai > and A 2 > 0. Without loss of generality, we further assume 



^ A2 — A2 

that yUl < yU 2 - 

Case 1: fii < r and fi2 < r. In this case, we have g(x, 1) = g(x, 2) = x and 

(£ — r)g(x, i) = jiiX — (Aj + r)x + XiX = (fii — r)x < 0, x > 0, % — 1, 2. 

Then Theorem 12.41 implies that V(x, 1) = V(x, 2) = x and that the optimal policy is to drive 
the process instantaneously to 0. 

Case 2: jii < r < ji 2 < where 

rAi + (r - ^i)(r + A 2 ) 



r + Ai - n\ 



(3.4) 



Note that £ > r. In this case, it can be shown that the unique solution to the system of 
coupled quasi variational inequalities 

max {(£ — r)0(x, a), 1 — (p'(x, a)} = 0, x > 0, a = 1, 2, 

is 

A 2 

0(x, 1) = x, 0(x, 2) = x. (3.5) 

v ; v 1 A 2 + r-/i 2 v y 
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Therefore Theorem 12. II implies that (f)(x,a) > V(x,a), a = 1,2. 

Next we show that there is a harvesting strategy Z* under which J(x, a, Z*) = <p(x, a) 
for all x > and a = 1, 2. To this end, we denote the harvesting region by % := (0, oo) x {1} 
and the continuation region by C := (0, oo) x {2}. Let the harvesting policy Z* be such that 
it drives the process instantaneously to the origin once the Markov chain enters state 1 or 
the process enters the harvesting region. Consequently, the extinction time is 



r 



inf {t > : a(t) = 1} = inf jt > : (X x ' a (t), a(t)) G ^} . 



One can verify that this harvesting policy and the corresponding harvested process satisfy 
all conditions in Theorem 12. 1[ part (b). In fact, it is obvious that J(x, 1, Z*) = x = <p(x, 1). 
Next we consider J(x, 2, Z*). Note that r = r x '' 2 has exponential distribution with parameter 
A2 and that 

X(t) = X x ' 2 (t) = xexp |(/i 2 - ~o|)i + (7 2 w(t)J , for all t G [0,r]. 

Therefore it follows that 

J(x, 2, Z*) = E Xj2 ! e~ rs dZ*(s) = E x , 2 [e- rT X(r)} 

= J e~ rt xexp |(/i 2 - ^a 2 2 )t + ^ 2 2 t| A 2 e- A2 'c/t 

= t — — ^ x = (j)(x, 2). 

A 2 + r - /i 2 

Hence V(x, a) = (fi(x,a) for all (x, a) G (0, 00) x {1,2} and Z* is an optimal harvesting 
strategy. 

Case 3: fii < r < £ < /i 2 , where £ is defined in (13 .4p . We claim that V(x, 1) = V(x, 2) = 
cxd for any x > 0. This is quite interesting. It indicates that even though fi± < r, we still 
have V(x, 1) = 00 thanks to the switching component a. In the sequel, we demonstrate that 
there exists an admissible harvesting policy Z under which J(x, 1, Z) and J(x, 2, Z) can be 
arbitrarily large and hence the claim follows. 

Fix some M > and define 77 := inf {t > : (X(t),a{t)) = (M,2)}. Note that the 
function u(x,i) = E 1V j[e~" ? ], < x < M, i — 1,2, solves the differential equation (£ — 
r)u(x,i) = 0. That is, u is a solution to the coupled system of differential equations 

-<j\x 2 u"(x, 1) + fiixu'(x, 1) — (r + \i)u(x, 1) + \iu(x, 2) = 0, 

2 (3.6) 

-ct 2 x 2 m"(x, 2) + fi 2 xu'(x, 2) - (r + A 2 )«(x, 2) + \ 2 u(x, 1) = 0. 

The characteristic equation of (13.61) is h{x) = gi(x)g 2 (a;) — AiA 2 = 0, where Qi{x) = \afx{x — 
1) + HiX — r — Aj, i — 1, 2. As argued in [9], h(x) has four real roots 0i > (3 2 > > f3 3 > /3 4 . 
Moreover, the condition [i\ < r < £ < jj, 2 implies that 1 > f3 2 > 0. Therefore u(x, i), a 
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solution of (13. 6p . can be written as 

4 

u(x, i) = E Xti [e- rv ] = C j xP *> < x < M, i = l,2 



i=i 

for some constants Cj, % = 1,2 and j = 1,2,3,4. As noted in [9], Cj = IjCj, where 
1 4 = nj \ \ ^~ ■ But as 

X(t) = xexp|y [fi(a(s)) — -a 2 (a(s))]ds + J a(a(s))dw(s) 

it follows that r\ — )■ oo a.s. as a; J, 0. Thus for % = 1, 2, we have E x . j[e~ rr '] -)• as 1 1 and 
hence 

Ea-ife-"'] = + C 2 x^ 2 , E Sj2 [e-^] = lid^ + l 2 C 2 x?\ 

where C\ and C 2 are constants, and 

Z A 2 _ giQgi) <Q ^ A 2 _ ^i(/3 2 ) ^ Q 

1 S-2(/3i) Ai ' 2 # 2 (/3 2 ) Ai 

Now the boundary conditions yield 

EM,i[e- r7? ] = c = dM* + C 2 M & , E M>2 [e-^] = 1 = l x C x M^ + Z 2 C 2 M^, 
where < c < 1. Solve the above equations for C\ and C 2 and we obtain 

n kc-l n I -he 



Notice that C 2 > 0. Consequently, we can write for x G (0, M) that 

^' lLe 1 (l 2 -h)M^ X + (l 2 -h)M^ X ' 

and 

E ^ 2[e l-kfc-ZOMA* +/2 (/ 2 -/ 1 )Mft :r • 
Now we choose Z(t) = MI[M,oo)x{2}(X(t),a(t)), t>0. Also, let 

fj := inf {t > : (A(t), a(t)) G [M, oo) x {2}} . 

Then we have rj < r] < r. Therefore the fact 1 > j3 2 > leads to 

J(x, 1, Z) = E Sil / e- rs dZ(s) > ME x>1 [e- rJ7 ] > ME^fe" 

(.'2 — n) ('2 — n) 

— > oo, as M — > oo. 
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Similar calculation shows that 

J(x,2,Z) > ll ^ C ~] ] x^M^ + ^iM^-A 
— > oo, as M — )• oo. 

The claim that V(x, 1) = V(x, 2) = oo thus follows. 

Case 4: fii > r and //2 > r. As in Case 3, we can show that the characteristic equation 
of (13. 6p h(x) = gx(x)g2(x) — A1A2 = has a solution < (3 2 < 1 and hence similar arguments 
as in Case 3 reveals that V(x, 1) = V(x, 2) = 00. 

Remark 3.2. In Theorem 12.11 part (b), condition (12.31) suggests that the optimal harvesting 
strategy Z will harvest only in the harvesting region 

■H = R + xM-C = {(x, a) e S xM: <f/(x, a) = f(x, a)} , 

which, in turn, implies that (f)(x, a) = J f(y, a)dy for (x, a) G H. Further, if we do harvest 
at time s (so X(s) 7^ X(s— )), the amount of harvest AZ(s) must satisfy condition (12. 5p . In 
other words, if we denote X(s— ) = x, a(s— ) = a, and AZ(s) = X(s—) — X(s) = 5x > 0, 
then we must have 

— f(x, a)5x = 4>(x — 5x, a) — 4>{x, a) = — / f(y,a)dy. (3.7) 

J x—8x 

However, if f(x,a) is strictly decreasing with respect to x for (x,a) G 7i, (\3.7\i can never 
be satisfied. In other words, there is no admissible optimal harvesting strategy at all. Then 
a natural question arises: Can we find an admissible e-optimal harvesting policy? In the 
following example, the answer to this question is positive. 

Example 3.3. As in Example 13. 1[ let the harvested process be given by (13. 2 p and the 
random environments be modeled by a two-state continuous time Markov chain a whose 
generator is Q. Our objective is to maximize the expected total discounted income from 
harvest 

V(x,a) = maxE Iia f e~ rs (l + X(s-))^dZ(s), (3.8) 
ZeA Jo 

where < 7 < 1, and r, r, and Z are as in Example 13.11 

Assume that ji\ > r and ^2 > r. As a result, the positive roots 

1 & fl ^ \ 2 2r 

of the equations ^afx(x — 1) + \iiX — r = satisfy < pj < 1, i = 1, 2. Suppose that 

Pi=P2 = V- (3-9) 

Note that there are many nontrivial examples (in the sense that \x\ 7^ fi2 and o\ 7^ a 2) where 
condition (13 .9p is satisfied. For example, if ^1 = 1, a\ = 2, \i2 = 2 — y/r, and a\ = 4, where 
< r < 1, then pi = p 2 = p = y/r. Under condition (I3.9p . we compute 

Hp) = 9i{p)92{p) - AiA 2 = (-Ai)(-A 2 ) - AiA 2 = 0, 
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where h,gi, and g% are defined in Example 13.11 Consequently, it follows that (£ — r)x p = 0. 
Assume 1 — p < 7 < 1. Detailed calculations reveal that 

f (i ^rv , if* g (o,&), 

0(x,l) = 0(x,2) = ^ ( / }1 _ 7 _ ( }1 _ 7 6(1 + fe) - 7 (3.10) 

- — — — + - — , if x e [b, +00), 

I 1-7 p 

solves the quasi- variational inequality 

max {(£ — r)(f)(x, a), (1 + x) -7 — 0'(x, a)} = 0, x > 0, a = 1, 2, 

where 6 = > 0. Therefore by virtue of Theorem 12. 1[ V{x 1 a) < 0(x, a) for all x > 

and a = 1,2. 

Next, we construct an e-optimal harvesting strategy Z £ . To this end, we denote the 
continuation region by C := (0, b) x {1, 2} and the harvesting region by "H := [b, 00) x {1, 2}. 
Let (x, a) G C and be the local time process for the process X at the point b. Then for 
the function (x, a) 1— > x p , we have 

/•Tjv rT N 

E Xia [e- rT «X(T N y] -x p = E Xj0t / e' rs (C - r)X(s) p ds - E X , Q / e- rs pX{s) p - l dL b {s), 

Jo Jo 

where T/v = r A iV A /?at as in the proof of Theorem 12.11 Thus it follows that 

pb^E^ / e- rs dL b (s) =x p - E Xta [e- rTN X(T N ) p ]. 
Jo 

Using (12. 6p . we can readily verify that E Xta [e~ rTN X(T N ) P ] ->• as N — > 00. Hence by letting 
N 00, we obtain 



X 1 ' 



E x , a I e~ rs dL b (s) = — . (3.11) 



Now let Z = L b . Then 



J(x, a, Z) = E x , a / e- rs (l + X( S ))^dZ( S ) = E x , a / e - rs (l + X (s))^ dL b (s) 
Jo Jo 

= (1 + b)-<E x , a / e -"dL 6 (s = 1 \ v \ x p 
Jo Pop 



-or = 01 x, a . 



Next, let (2, a) GK. If x = 6, define Z 1 £ A such that Z x (0) = AZ^O) = g > and 
Z\t) = L b (t) for t > 0. Then 

J(x, a, Z 1 ) = (1 + &) _7 £ + J(6 — g, a, L b ) 

= d +»r e+^^ib-.r (3 . 12) 

6(1 + 6)~ 7 . , 

> - s 3 = (j)(x, a) - e/3 

p 
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for q sufficiently small. 

If x > b. Then as in the proof of Theorem EH for n sufficiently large, let 8 = q = n~ 5 , 
and ti = iq/n for i — 0, 1, . . . , n — 1. Let Z 2 E A increase only at times £j, i — 0, 1, . . . , n— 1. 
More specifically, Z 2 (t ) = AZ 2 (t ) = 5 and for i — 1, 2, . . . , n — 1, 

AZfc) = (l(t-) -(*-(« + 1)5)) + . 

Denote # := inf jt > : X(t) < b\. Note that 9 < t n - X < n' 5 and X(6) = b a.s. Define 

Z%t) = Z 2 (t)I {t<e} + Z\t)I {t > 0} . (3.13) 

It is easy to verify that for any x, y > 0, we have 

|(l + x)- 7 -(l+y)~ 7 | <\x-y\, 

and hence / is Lipschitz continuous. As a result, similar arguments as those in the proof of 
Theorem 12.41 reveal that 

E x>a J e-"(l + X{s-))^dZ 2 {s) > J\l + y)^dy - | (3.14) 

by choosing n sufficiently large. Then for n sufficiently large, we have from (13.121) and f)3.14p 
that 

J{x, a, Z e ) = E x , a [ e- rs (l + X(s-))^ 'dZ £ (s) 

= E x , a [ e~ rs (l + X(s-))-~<dZ 2 (s) + E x , a f ~ e- rs {l + X{s-))-idZ l {s) 
Jo Je 

> E x , a J e- rs (l + X{s-))-'dZ 2 {s) + E x , a [e~ re ( J(b, I, Z 1 ) A J(b, 2, Z 1 ))] 

> (i +z) ^_ (1+t) ^ + b{1+b) -?_ e = _ e 

1-7 p 
This shows that the harvesting policy Z £ is e-optimal. 

4 Properties of the Value Function 

Theorem 12.11 gives sufficient conditions for a function to coincide with the value function. 
In particular, the function must satisfy the system of quasi- variational inequalities ( 12. ip . 
It is natural to ask whether the converse is true: "Does the value function V defined in (jl.8p 
always satisfy (12. ip ?" In general, the answer is no since the solution V is not necessarily 
smooth enough. An alternative definition for a solution to the quasi-variational inequalities 
( 12. ip is that of a viscosity solution (see Fleming and Soner [S]). 

Therefore this section is devoted to the properties of the value function V. We present 
sufficient conditions under which the value function V is continuous. Also, we show that V 
is a viscosity solution to the coupled system of quasi variational inequalities ( 12. ip . 
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4.1 Continuity 



As we indicated in Section [TJ in the definition of J in (II .7p . both the extinction time r and 
the harvesting strategy Z may depend on the initial condition X(0— ) = x. Consequently the 
standard arguments for continuity of V using Lipschitz continuity and Gronwall's inequality 
(as in j36l Propsition 4.3.1]) or Dini's theorem (as in [HI Lemma V.2.1]) do not apply here. 
In the sequel, we establish the continuity of V by first studying some elementary properties 
of V. 

The following lemma is immediate, which asserts that the value function V(x, a) is non- 
decreasing with respect to the x variable. 

Lemma 4.1. For each a G M. and any < y < x, we have 

V(x, a) > f(x, a)(x -y) + V(y, a). (4.1) 

Proof. Fix some a G Ai. If y < x, then for any harvesting strategy Z G A, we define Z to 
be the harvesting strategy such that Z(t) = Z(t) + (x — y) for any t > 0. It is obvious that 
Z G A. Note also that AZ(t) = AZ(t) for all t > 0. Let X denote the process satisfying 
(11. 4p with initial condition X(0— ) = y and harvesting strategy Z. Similarly, X denotes the 
process satisfying (11. 4p with initial condition X(0—) = x and harvesting strategy Z. Then 
we have X(t) = X(t) for all t > 0. Consequently, it follows that 

J(x,a,Z) = f(x,a)(x-y) + J(y,a,Z). 

Since V(x, a) > J(x, a, Z), we have 

V(x,a) > f(x,a)(x - y) + J(y,a,Z), 

from which (14. ip follows by taking supremum over Z G A. □ 

Lemma 4.2. For each aeM and any < y < x, we have 

V(x,a) < V(y,a) + max{V(x -y,j) : j = l,...,m}. (4.2) 

Proof. Let x > y > 0. We consider a harvested process X with initial conditions X(0— ) = 
x,a(0) = a and harvesting strategy Z G A. Define 9 := inf{t > : X(t) < x — y} and 
t := inf{t > : X(t) £ S}. Then r > 9. 

Case 1: 9 = oo. Note that r = oo. Let ?/ G (0, x) and X be an another harvested process 
with initial conditions X(0— ) = y, a(0) = a and harvesting strategy Z G A, where we choose 
Z(t) = Z(t) for all t > 0. Consequently, it follows that < X(t) = X(t) - (x - y) < X(t) 
for all t > 0. Using the assumption that /(■, a) is nonincreasing for each a G Ai, we have 



J(x,a,Z)=E [ e- rs f(X(s-),a(s-))dZ(s) 
Jo 

<E [ T e- rs f(X(s-),a(s-))dZ(s) 



(4.3) 



J(y,a,Z) < V(y,a) 
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Case 2: 9 < oo. Then we can write 



J(x,a,Z)=B / e- rs f(X(s-),a(s-))dZ(s) 



E 

+ 



e-™f(X(s-), a(s-))dZ(s) + e' r6 'f(X(9-), a{9-))AZ{9) 



[0,0) 



e- rs f(X(s-),a(s-))dZ(s) 



As in Case 1, we consider y G (0,x) and another harvested process X with initial conditions 
X(0— ) = y,a(0) = a and harvesting strategy Z G i, where we choose Z(t) = Z(t) for all 
< tj: 9 and Z{9) = Z{9-) + X{9-). As a result, X(t) - X(t) = x - y for all < t < 9 
and X{9) = 0. Then it follows from the monotonicity of / that 



J(x,a,Z) < E 



e- rs f(X(s-), a(s-))dZ(s) + e~ r0 'f(X(9-), a(9-))AZ(9) 



[0,6) 



< E 



+ e- re f(X(9-), a(9-))[AZ(9) - AZ(9)] + / e~ rs f(X(s-), a{s-))dZ{s) 

J(0,t] 

e- rs f(X(s-),a(s-))dZ(s) + e~ re f(X(9-),a(9-))AZ(9) 

[0,8] 



+ / e- rs f(X(s-),a(s-))dZ(s) 

J(B,r] 

J(y, a,Z) + E e- rs f(X(s-), ot{s-))dZ(s). 



Note that X{9) < x — y. Therefore by virtue of Lemma [4. 1[ we have 



E 



e~ rs f{X(s-),a{s-))dZ(s) = E 



E 



e- rs f(X(s-),a{s-))dZ{s)\F e 



<E[e- re V(X(9),a(9))] 

< max{V(x - y,j),j = 1, . . . ,m} 



Hence it follows that 



J(x,a,Z) < J(y,a,Z) + max {V{x - y,j),j = l,...,m} 
< V(y,a) + max {V(x -y,j),j = l,...,m}. 

Combining cases 1 and 2, we conclude that for any Z G A, we have 

J(x,a,Z) < V(y,a) +max{V(x - y,j),j = 1, . . . , m} . 

Now (14.21) follows by taking supremum over Z G A. 

For /i > 0, we denote 

C h := inf {t > : A x ' a (t) = /i} , where A x ' a (t) is the solution to (TO) . 



(4.4) 



□ 



(4.5) 
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Lemma 4.3. Suppose that for any t > and h > 0, we have 



,. ,. >. max X(s) < h > — > 1, asxiO, for each a E Ai. 

0<s<( M 



If V(x , a) < oo /or some x > and ever?/ a£M, i/iere 

limV^x, a) = 0, for each a E Ai. 



(4.6) 



(4.7) 



Proof. It follows from Lemmas 14.11 and 14.21 that V(x,a) < oo for all (x,a) E S x .M. Fix 
some (x, a) G M + x .M and 2 G A For any h > and £ > 0, let 77 := r A C/i A £, where r 
is the extinction time defined in (II .6p and C/i in fl4.5[) . Then n < r and 77 < r if and only if 
r > ( h At. Note that (14. 6 j) implies that P XjCe {Co < C/i A ~~ 1) as x 4- 0, for each a G Ai. 
Hence for any e > 0, we can find a. 5 > such that 



P x ,a {Co < (h A £} > 1 — s, for any < x < 5 and each a E Ai. 



(4.8) 



Note that for any s > 0, X x,a (s) < X x,a (s) by the admissibility of Z G A. Hence it follows 
that r < Co- This, together with (14.81) . implies that P Xta {r < ( h A £} > 1 — e. That is 



P x ,a {t > v} < e 5 f° r an y < x < 5 and each a G .M. 
On the other hand, by virtue of [351 Proposition 2.3], we have 



Now we compute 



sup X(s) 

0<s<CoAt 



< e, for any < x < 5 and each a E Ai. 



(4.9) 



(4.10) 



J(x,a,Z) =E X , Q / e- rs f(X(s-),a(s-))dZ(s) 
Jo 

E,„ / e- r 7(X(s-),a(s-))dZ(s) 
'0 



+ E Xj( 
:= A + fi. 



'{t>,} / e~ r 7(X( S -),«( S -))dZ( S ) 



Since /(•, a) is nonincreasing and that X(s-) > for all < s < rj, it follows that 
f{X(s-),a(s-)) < f(0,a(s-)) < max/(0,i) = X, 

where in the above and hereafter, K is a generic positive constant not depending on x 
or £ whose exact value may change in different appearances. Hence it follows that A < 
K~E Xt(X [Z(j])]. Further, since Z is an admissible harvesting strategy, we have 

Z(v) < X(t]) < sup X(s). 

0<s<CoM 
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Note in the above, we used the fact that 7] = TA(h/\t<TAt<(oAt. Thus we have from 
fTCTOj) that A < E x>a [Z(r))] < Ke. 

On the other hand, 



I{r>r,} 



<f(X(s-),a(s-))dZ(s) 



I {T>V} B / e- rs f(X(s-),a(s-))dZ(s)\F ri 



B = E Xj0l 

= E 

<E,, Q [i^e-^ViX^air]))]. 
By the definition of 77, we have X(n) < X(n) < h. Thus Lemma [4.11 leads to 

V(X(r]),a(rj)) < V(h,a(r])) < m&xV(h,i) < 00. 

Therefore it follows from the above computations and (14.91) that 

J(x, a, Z) < Ke + max V(h, i)P x a {r > r/} < Ke. 

Taking supremum over Z E A, we obtain V(x, a) < Ke, for all < x < 5 and each a e M.. 
Therefore (14 .7p follows and this completes the proof of the lemma. □ 



By virtue of Lemmas I4.1[ 14. 2\ and I4.3[ we have the following theorem, which presents a 
sufficient condition for continuity of the value function. 



Theorem 4.4. Let the conditions of Lemma 14.31 be satisfied. Then the value function V 
defined in (II. 8ft is continuous with respect to the variable x. 



Remark 4.5. Note that (14. 6 p is the crucial assumption in Theorem 14 .4| it also plays a key 
role of the proof of Lemma [4.31 One may wonder under what condition(s), is (14 .6p valid? 



Example 4.6. If the unharvested process is given by 

dX{t) = b(a(t))dt + a(a{t))dw{t), 



(4.11) 



where for each a e A4, b(a) £ R and a(a) > 0, then using the same argument as that of [3 
Lemma 1], we obtain (14. 6p . 

Next we present a sufficient condition for ( 14. 6p . 
Proposition 4.7. // there exists a function W : 5* x Ai t— > M + satisfying 

(i) for each a £ M., W(-, a) is continuous on [0, 00) and vanishes only at x = 0, 

(ii) CW(x, a) < for all (x,a) £ S x M. 
Then for any t > and h > 0, we have 

P x a i max X(s) < h > — > 1, as x 1 0, for each a £ Ai. 
0<s<( At w J ' * ' J 
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Proof. Fix some t > and h > 0. Let (x, a) G S x M with x < h. Denote := 
inf {W(y, j) : £/ G S, y > h, j G M.}. Then assumption (i) implies that Wh > 0. By virtue of 
Ito's formula and assumption (ii), we have 

E x>a W(X(tA( A( h ),a(tA( A( h )) = W(x, a) + E x>a / CW(X(s),a{s))ds 

Jo 

< W(x,a). 

Now since W is nonnegative, it follows that 

W(x, a) > E x , a [W(X(Ch), a(Cfc))% h <CoAt}] > W^P a ,« < Co A t} . 
Thus we have 

^ *. W(x, a) 

Px,« {a < Co a t} < 



w h 

This, together with assumption (i), leads to 

W(x a) 

Px,« {Co A t < G} > 1 -»• 1, as x | 0. 

Note that Co A t < C/i if and only if max < s <( oA< < /i. Therefore the desired assertion 
follows. □ 



4.2 Viscosity Solution 

In this subsection, we aim to characterize the value function as a viscosity solution of the 
coupled system of quasi- variational inequalities (12.11) . Let's recall the notion of viscosity 
solution. 

Definition 4.8. A function u is said to be a viscosity subsolution of (12.11) . if for any (xq, «o) G 
S x Jvt and function (p(-,a) G C 2 (S) satisfying (p(x ,ao) = u(x ,a ) and ip(x,a) < u(x,a) 
for all x in a neighborhood of Xq and each a G A4, we have 

max {(£ - r)^(x , a ), /(x , a ) - <f'(x Q , a )} < 0. 

Similarly, a function u is said to be a viscosity supersolution of (12. lft . if for any (xo,ao) G 
SxM and function <p(-,az) G C 2 (S) satisfying <p(xo,a ) = u(x ,a ) and <^(x,a) > u(x,a) 
for all x in a neighborhood of Xq and each a G .M, we have 

max {(£ - r)<^(x , a ), f(x , a ) - (p'(x , a )} > 0. 

The function u is said to be a viscosity solution of (12.11) . if it is both a viscosity subsolution 
and a viscosity supersolution. 

Theorem 4.9. Assume the conditions of Theorem 14.41 T/ien t/ie value function V is a 
viscosity solution of the coupled system of quasi-variational inequalities (12. ip with boundary 
condition (14.71) . 
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Proof. The proof is motivated by [5] and [8, Theorem VIII 5.1]. We divide the proof into 
two parts. The first part shows that V is a viscosity subsolution of (|2.ip . while the second 
part establishes that V is viscosity supersofution of (12. ip . 

Step 1. We show that V is a viscosity subsolution of (12. ip . That is, for any (xo,«o) G 
SxA^ and any C 2 function </>(•, •) satisfying (p(xo, ao) = V(xo, ao) and that (f>(x, a) < V(x, a) 
for all x in a neighborhood of and each a G M, we have 



max{(£-r)0(xo,ao),/(a:o,ao) _ 4>'( x a, a o)} < 0. 



(4.12) 



Let B £ (xo) := {s G R : |a; — £o| < e}, where £ > is sufficiently small so that (i) B E (x ) C S 1 
and (ii) 0(x, a) < V(x, a) for all (x, a) G -B £ (x ) X.M, where B e (x ) = {x G R : \x — x \ < e} 
denotes the closure of B £ (xq). Choose Z G A such that Z(0—) = and Z(t) = r] for all 
t > 0, where < rj < e. Let X(-) = X xo,a °(-) be the corresponding harvested process with 
initial condition (xo,«o) anc ^ harvesting strategy Z(-). Put 

:= inf jt > : X(t) ^£ £ (x )}- 

Note that the chosen harvesting strategy Z guarantees that X(-) has at most one jump at 
t = and remains continuous on (0,6]. This, together with the choice of e, implies that 
9 < t and that X(t) G B £ (xq) for all < t < 6. By virtue of the dynamic programming 
principle (II. 9p . for any h > 0, we have 



(f)(x ,a ) = V(x ,a ) 

> E / 

.Jo 

« /' 

> e' 



e- rs f(X(s-), ot{s-))dZ(s) + e~ r{eAh) V(X(9 A /i), a(0 A h)) 
e- rs f(X(s-),a(s-))a!Z(s) + e^ ^ (j){X{6 A h), a{6 A /i)) 



(4.13) 



Applying the generalized Ito formula to the process e rs (p(X(s), a(s)), we obtain 
e- r{8Ah) <f)(X(6 A h), a{9 A /i)) - 0(x o , a ) 

e- rs {C-r)(j){X{s) ) a{s))ds+ / e" r y(X(s), a(s))a(X(s), 



o 







6/\h 



e- rs / (X(s),a(s))rfZ c (s) + 



<f>(X(s),a( S -))-<t>(X(s-),a(s-)) 



o<s<e/\h 

Since G C 2 , a is continuous, and X(s) G -B e (xo) for all < s < 6, it follows that 

p0Ah 

E / e- rs (p'(X(s),a( y s))o-(X( y s),a( y s))dw(s) = 0. 
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Consequently, we have 

<f>(x , a ) = Ee- r(6Ah) <P(X(9 A h), a(9 Ah)) — E / e~ rs (C - r)(f>{X(s), a(s))ds 

Jo 



+ E 



6 Ah 



e- rs <f)'(X(s),a(s))dZ c (s) 



(4.14) 



-E ]T e-"[0(X( S ),a( S -))-0(X(s-),a( S -)) 

0<s<6Ah 

A combination of (I4.13P and (I4.14p leads to 

i-OAh p0Ah 

0>E/ e- rs f(X(s-),a(s-))dZ(s) + B / e~ rs (C - r)(f>(X(s), a(s))ds 
Jo Jo 

r0Ah 

-E / e- rs (f)'(X(s),a(s))dZ c (s) 
Jo 



(4.15) 



+ E £ 



<f>(X{s),a(s-))-<f>(X{s-),a(s-)) 



0<s<9Ah 

Now let r] — 0, i.e., = for any t > 0. Then (I4.15P can be rewritten as 



> E 



0Ah 



e- rs (C-r)4>(X(s),a(s))ds 



= E / e- rs (C~r)(l ) (X(s),a(s))I {s <e}ds. 
Jo 

Note that e~ rs (£ — r)<p(X(s), a(s))/{ s <0} is bounded for all < s < h by the definition of 9. 
Hence there exists some < £ h < h such that 

> E I* e- rs (C - r)<P(X(s),a(s))I {s< 8 } ds > KE[e~^ h {C - r)0(X(e h ), a(£ h ))% 1<e} ]. 
Jo 

Note that as h 4- 0, £ ft 4- 0. This implies that (X(£ ft ), a(^ h )) — >■ (:r , «o) a -s. due to the choice 
of Z = 0. With the continuity of (£ — r)0, we conclude that 

e- r t h (C-r)<f)(X(t h ),a(t h ))I {e < 0} ->■ (£ - r)0(x o , a ), a.s. as /i | 0. 



Therefore it follows from the bounded convergence theorem that 

(£ - r)0(a; o ,a o ) < 0. 
On the other hand, if we choose < rj < e, then (I4.15P reduces to 



(4.16) 



OAh 



E / e rs (C - r)(f)(X(s), a(s))ds + f(x , a )r) + (f)(x - rj, a ) - <p(x , a ) < 0. 
Jo 

Now sending h 4 0, we have 

f(x , a )r] + 0(x o - i], a ) - <j>(x , a ) < 0. 
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Finally, divide the above inequality by r] and let 77 — >■ 0, it follows that 

f{x ,a )-<t>'{x ,a ) <0. (4.17) 

Now ( 14.121) follows from a combination of (14.161) and (14. 17ft . 

Step 2. We need to show that V is also a viscosity supersolution of (12. ip . That is, 
for any (x ,a ) G S x M. and any ip G C 2 such that (p(x ,ao) = V(i ,«o) and that 
<p(x , a ) > V(x , a ) for x in a neighborhood of x and each a e M, we have 

max {(£ - r)(p(x , a ), f(x , a ) - (f'{x , a )} > 0. (4.18) 

Suppose on the contrary that (14.181) was wrong, then there would exist some (x , a ) E SxAi, 
a ip G C 2 , and a constant A > such that 

max {(£ - r)(p(x , a ), f(x , a ) - ^'(^0, "o)} < -2 A < 0. (4.19) 

In what follows, we will derive a contradiction to (I4.19p . This is achieved in several steps. 
First we use the generalized Ito formula and (I4.20p to obtain (14.211) . Next, detailed analysis 
using the monotonicity of the functions V and / leads to (14.26)) . Then we claim in (14.271) 
that the last term in (14.26)) is bounded below by a positive constant, from which, with the 
aid of dynamic programming (11.91) . we obtain a contradiction to (I4.19p . The final step of the 
proof is devoted to the proof of (14.271) . 

Fix some Z G A and let X(-) = X x °' a °(-) be the corresponding harvested process. Define 
B e (xo) as in Step 1, where e > is small enough so that (i) B £ (xo) C S, (ii) <p(x, a) > V(x, a) 
for all (x,a) G B £ (x Q ) x M, and (iii) 

max {(£- r)(p(x, a), f(x, a) - ip'(x, a)} < -A < 0, (x, a) G B £ (x ) x M. (4.20) 

Let 9 := inf |t > : X(t) ^ i? e (x )|. Then 9 < r. It follows from the generalized Ito 
formula that 

Ee- re ^(X(9-),a(9-)) - <p(x ,a ) 

t-e- i>e- 
= E/ e- rs (£-r)<p(X(s),a(s))ds-E / e- rs ip'(X(s), a(s))dZ c (s) 
Jo Jo 

+ e~ rS Ux(s),a(s-))-^(X(s-),a(s-)) 



o<s<e 



Note that 



p(X(s),a(s-))-p(X(s-),a(s-)) 

= (X(s) - X(s-W(X(s-) + z(X(s) - X(s-)), a(s-)) 
= -AZ(s) V '(X(s-) + z(X(s) - X(s-)), a(s-)) 
for some z G [0, 1]. But by virtue of (14.20)) . for all < s < 9, we have 

-<p'(X(s-) + z(X(s) - X(s-)), a(s-)) < -f(X(s-) + z(X(s) - X(s-)), a(s-)) - A. 
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Further, since X(s) < X(s—) + z(X(s) — X(s—)) < X(s—) and /(•,«) is non-increasing, we 
have 

-f(X{s-) + z(X(s) - X(s-)), a(s-)) < -f(X(s-), a(s-)). 
Hence it follows from ( I4.20p that 

Ee- re tp(X(6-),a(6-))-tp(x Q ,a ) 

t-6- n6- 

< E / e~ rs {-A)ds + E / e- rs (-f(X(s), a(s)) - A)dZ c {s) 
Jo Jo 

+ e- rs (-f(X(s),a(s))-A)AZ(s) 

0<s<6 

p9- p6 r6- 

= -E/ e- rs f(X(s),a(s))dZ(s)~ AE e' rs ds- AE e - rs dZ(s). 
Jo Jo Jo 

Therefore 

<p{x , a ) > Ee~'\(X(9-), a{9-)) + E / e - rs f(X(s-), a{s-))dZ{s) 

Jo 



+ AE 



e~ rs ds+ / e~ rs dZ{s) 



(4.21) 



'0 jo 

Note that X(9) < X{9-) and X{9-) G B £ (x ). Thus there exists some A e [0, 1] such that 

x x ■= X(9-) + X(X(9) - X(9-)) = X{9-) - \AZ(6) e dB e {x ). 

Moreover, X(9) < x x < X(9-). Note that 

<p(X(9-), a(6-)) - ip(x x , a(6-)) = (X(9-) - x x )f'(X(9-) + z(X(9) - x x ), a(6-)) 

= XAZ(9)ip'(X(9-) + z(X(9) - x x ), a(9-)). 

But f)4.20p and the monotonicity of f(-,a) imply that 

<p'(X{9-) + z{X{9) - xx), a(9-)) > f{X{9-) + z(X(9) - x x ), a(9-)) + A 

>f(X(9-),a(9-)) + A. 

This, together with the fact that AZ(8) > 0, leads to 



<p(X{9-), a(9-)) - y?(x A , a(9-)) > XAZ(9) [f(X(9-), a(9-)) + A 

Combing (fl~2H and P~22l) . we obtain 

V(x Q ,a ) = ip(x ,a ) 

>E / e- rs f(X(s-),a(s-))dZ(s) + Ee- rd <p(x x ,a(9-)) 
Jo 



(4.22) 



+ AE 



e~ rs ds+ / e- rs dZ(s) 

(! Jo 



+ XEe- r8 AZ(9) f(X(9—), a(9-)) + A . 

(4.23) 
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Note that x\ G B e (xo). Therefore we have <p(x\, a(9— )) > V(x\, a{6— )). On the other 
hand, since X(9) < X\, it follows from (14. ip that 



V(z A , a(9-)) > V{X(9), a(9-)) + [x x - X(9)]f(x x , a{9-)) 

> V(X(9), a{6-)) + (1 - \)AZ(9)f(X(9-), a(9-)). 



(4.24) 



Note that 

Ee- re V(X(9), a{9-)) = Ee- re V(X(9), a{9)). (4.25) 

In fact, by virtue of [35| Theorem 2.12], a(-) is continuous in mean square. Hence it follows 
that 

Ee- r9 V(X(9), a{9-)) - Ee- r9 V(X(9), a(9)) 



< E 



e- re [V(X(9),a(9)) - V(X(6), a(9-))]I {a(e ^ a(e _ )} 



< KP {a{9) ^ a{9-)} = 0, 

where K is some positive constant. Therefore (14.251) follows. Now put ( 14.241) and ( 14.251) into 
( I4.23P and we obtain 



V(x , a ) > E / e- rs f(X(s-), a{s-))dZ{s) + Ee- rd V(X(9), a{9)) 
Jo 



'0 

+ AE 



e~ rs ds+ / e- rs dZ(s 
o Jo 

+ XEe- re AZ(9) \f{X(9-), a{9-)) + A 

E I e- rs f(X(s-),a(s-))dZ(s) + Ee- r0 V(X(9),a(9)) 
r e- 

e~ rs ds+ / e- rs dZ{s) + \e- re AZ(9) 
ii Jo 



+ (1 - X)Ee- rd AZ(9)f(X(9-), a{9-)) 



■)o 
+ AE 



We now claim that for some constant k > 0, we have 



E 



e~ rs ds+ / e~ rs dZ(s) + \e~ r0 AZ{9) 
Uo Jo 



> K. 



(4.26) 



(4.27) 



Assume ( I4.27P for the moment. Then ( I4.26P can be rewritten as 

r e 

V{x , a )>E e- rs f(X(s-), a{s-))dZ{s) + Ee" r V(X(#), a{9)) + Ak. (4.28) 
Jo 



Taking supremum over Z & A, it follows that 



V(x , a ) > sup E 



s f(X(s-), a(s-))dZ{s) + e- r "V(X(9), a(9)) 



+ Ak. (4.29) 



30 



But in view of the dynamic programming principle fll.9p . (I4.29p can be rewritten as 

V{x , a ) > V(x , a ) + An > V(x , at ). 
This is a contradiction. So we must have (I4.18p and hence V is a viscosity supersolution of 

Now it remains to show (14. 2 7ft . To this end, we consider the function W(x,a) := 
\x — xo| 2 — s 2 for (x, a) G B e {xq) x Ai. Then it follows that 

(C — r)W(x, a) = 2{x — x )b(x, a) + -2<j 2 (x, a) — r(\x — x \ 2 — e 2 ). 
Since W, 6, and a are continuous, and Ai is a finite set, it is obvious that 



(£-r)W(x,a) 



< K < oo 



(4.32) 



for some positive constant K. Now let K := anc ^ define W(x, a) = K W(x, a) for 

(x,a) G B £ (xq) x Ai. Then it follows immediately that 

\{£-r)W{x,a)\ < 1, (z, a) G 5 £ (x ) x M. (4.30) 

Moreover, we have 

W'(x,a) = 2K (x-x )>-l. (4.31) 
Now apply the generalized Ito formula to e~ rs W (X (s) , a(s)) , 

E[e- re W(X(9-), a(9-))) - W(x , a ) 

p6- pQ- 

= E e~ rs (£-r)W(X(s),a(s))ds-E e~ r W(X(s), a(s))dZ c (s) 
Jo Jo 

+ e- ra [W(X{s),a{s-)) -W(X{s-),a(s-))}. 

o<s<e 

But by virtue of (14. 31ft . we have 

W(X(s),a(s-)) - W(X(s-),a(s-)) 

= W'(X(s-) + z(X(s) - X(s-)), a(s))(X(s) - X{s-)) 
= -W'{X{s-) + z(X(s) - X(s-)), a(s))AZ(s) 
< -AZ(s). 

Hence it follows from (OD)) - (|Pgj) that 

E[e- rd W(X(6-), a{e-))\ - W(x , a ) 

p6 p6- 

<E e~ rs ds + E / e- rs dZ c (s) + E ^ e" rs AZ(s) 

Jo Jo n^^n 



(4.33) 



O<s<0 



(4.34) 



p6 p6- 

E / e~ rs ds + E / e~ rs dZ{s). 
Jo Jo 



31 



Also, recall that X{9) <x x < X(9-). It follows from (jOTjl that 



W(X(9-), a(6-)) - W(x x , a{6-)) 

= W\x x + z(X(9-) - x x ), a(9-)) \x(9-) - x x 

= \W\x x + z(X(9-) - xx), a(6-))AZ(0) 
> -XAZ(9). 



(4.35) 



Combining ( 14.341) and CI4.35f) . we have 



■e 



E / e~ rs ds + E / e~ rs dZ(s) + \Ee- r6 AZ(9) > Ee- r6 W(x x , a(8-)) - W(x , a Q ). 



But xx € dB e (x ). Consequently, W(xx, a{6—)) = 0. Also, it is immediate that W(x , a ) = 
—KqE 2 . Hence it follows that 



5 Conclusions and Remarks 

In this work, we considered the optimal harvesting problem for a single species living in ran- 
dom environments. We first established a verification theorem, based on which we explicitly 
constructed an e-optimal harvesting strategy under additional conditions. Next we obtained 
the continuity of the value function and further characterized it as a viscosity solution of the 
coupled system of quasi- variational inequalities (12. ip . 

In examples 13.11 (cases 1 and 2) and 13.31 thanks to the special structures of the harvest- 
ing and continuation regions, we were able to obtain the value functions and (e-)optimal 
harvesting policies. It will be very interesting to investigate whether (e-)optimal harvesting 
policies exist in more general settings. 

The next logical step is to consider optimal harvesting strategy for multiple but finite 
number of interacting species in random environments. For virtually all ecosystems, the 
species often interact with each other and a small change of one population may have sig- 
nificant effects on other populations. Therefore to apply the mathematical findings in real 
worlds, one must consider the interactions among the species in the ecosystem. It seems that 
some results of this paper can be extended to multiple interacting species. For example, a 
verification theorem like Theorem 12.11 can be established using almost the same argument. 
But one may no longer get closed-form value functions and optimal controls by solving the 
corresponding quasi-variational inequalities. Also, we may be able to show that the value 
function is a viscosity solution of the quasi-variational inequalities but it is not immediate 
to identify condition(s) under which the value function is continuous. 



E 




e- rs dZ(s) + XEe~ rd AZ(9) > K e 2 = k > 0. 



This establishes ( 14.27ft and hence finishes the proof of the theorem. 



□ 



32 



In view of (TTJ Section 4], within the same framework and the same optimality criterion 
considered in this paper, we may consider a more general problem where the controlled state 
process is given by 

dX(t) = b(X(t), a(t))dt + a(X(t), a(t))dw(t) - ^(X(t-), a(t-))dZ(t), 

where X, Z £ ¥L n , 6,(7,7 are suitable functions with appropriate dimensions, and w is an 
n-dimensional standard Brownian motion. Since every process of bounded variation can be 
written as a difference of two nondecreasing processes, the control space can be enlarged by 
allowing the singular control Z to be an adapted process with bounded variation. 

A number of other questions deserve further investigations. In particular, in many prac- 
tical situations, it is virtually impossible to obtain the explicit form of the value function 
and an optimal control by solving (12.11) . Therefore a viable alternative is to employ numeri- 
cal approximations. The controlled Markov chain approximation method developed in [16] 
seems promising. We may also consider relaxed control, under which we may achieve the 
optimal value with an optimal control. Another problem of great interests is to consider the 
case when the random environment or the Markov chain a is unobservable. 
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